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(57) Abstract: 

PROBLEM TO BE SOLVED: To 
provide a, display control device 
capable of automatically detecting the 
natural movement of a user of display 
trying to search a display element 
displayed on a display, changing the 
display state of the display element 
and enabling the user to easily 
confirm the display element. 

SOLUTION: When a user sits down 
in the front of a display 3, the upper 
half of the user's body and its 
background are photographed by a 
CCD camera 1, its image is 
temporarily recorded in an image 
memory part 1 1, an image of a face is 
extracted by an image feature 
extraction part 12 and recorded in an 
image feature memory part 18 and 
the center of gravity of the extracted 
face image is calculated by a center- 
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of-gravity calculation part 14. Then, 
the center of gravity obtained at the 
time of movement is divided by a 
prescribed time and recorded in a 
center-of-gravity time series memory 
part 15. A moving distance judging 
part 16 compares the contents of the 
memory 15 with that of a reference 
data part 19, judges whether the 
display stkte of the display element 
displayed on the display 3 is to be 
changed or not, and in the case of 
changing :the display state, a display 
control part 17 changes the display 
state and displays a picture whose 
display state is changed on the 
display 3. j 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

l.This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] The display control unit carry out having an image pick-up means picturize the location in 
which the user of a display is located, an extract means extract a specific image from the image 
picturized and obtained, a judgment means judge the migration condition of the extracted image, and a 
means change the display condition of the display element displayed on the display based on the 
judgment result of a judgment means as the description. 

[Claim 2] Said extract means is a display control unit according to claim 1 characterized by extracting 
some bodies as a specific image. 

[Claim 3] Said judgment means is a display control unit according to claim 1 or 2 which makes it a 
criterion for the extracted specific image to have moved to the position of an image field. 
[Claim 4] Said judgment means is a display control unit according to claim 1 or 2 which makes it a 
criterion for the movement magnitude per fixed time amount of the extracted specific image to exceed 
the specified quantity. 

[Claim 5] The display control unit according to claim 1 to 4 characterized by the ability to change the 
display condition of a display element according to the migration direction of a specific image. 
[Claim 6] Two or more display elements displayed on the display are display control units according to 
claim 1 to 5 characterized by changing a display condition so that it can carry out visible [ of the name 
display of a display element ]. 

[Claim 7] The display control unit according to claim 1 to 6 which will be characterized by changing the 
display condition of a display in said extract means if it becomes impossible to extract a specific image. 
[Claim 8] The display control unit according to claim 1 to 7 characterized by displaying the image 
picturized with the image pick-up means on a display. 

[Claim 9] The display control unit according to claim 1 to 8 characterized by making the image which 
picturized with the image pick-up means and was displayed on the display into the icon for starting of a 
means which changes the display condition of a display element. 

[Claim 10] The display control approach characterized by changing the display condition of two or more 
display elements which judged that the image which extracted the specific image and was extracted 
moved from the picturized image data, and were displayed on the display based on the judgment. 
[Claim 11] The record medium carry out having recorded the display control program included in the 
processing which picturizes the location in which the user of a display is located, the processing which 
extract a specific image from the image data picturized and obtained, the processing which judge that 
the extracted image moved, and the processing which change the display condition of two or more 
display elements displayed on the display based on the judgment result as the description. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates modification of the display condition of display elements, 
such as two or more windows, icons, etc. which are displayed on a display, to the interface for multi- 
window display controls recognized and controlled in migration of the body. 
[0002] 

[Description of the Prior Art] When working in multi-window control units, such as a personal computer 
(a personal computer is called hereafter.), it works to coincidence using two or more software in many 
cases. In that case, it works by displaying two or more windows on one display. When two or more 
windows are displayed on a display, since a window hides depending on overlap and the case, it may not 
understand anymore what kind of window exists on a display only by glancing at a screen. Drawin g 15 
is an example which shows the condition of the window displayed on the display usually used. The title 
bars 71b-74b which are the name displays which display the name of a window along with one side of 
the upper part of each windows 71-74 are formed. The polymerization of each window is carried out to 
the sequence of 71, 72, 73, and 74 now. In a window 73, since the window 71-72 is carrying out the 
polymerization, title bar 73b of a window 73 cannot be checked. Moreover, in a window 74, since the 
window 72-73 is carrying out the polymerization, title bar 74b of a window 74 can check only a part. 
[0003] The command of screen arrangement is chosen using the mouse which is an input unit by making 
into the approach of learning easily what kind of window to display on the display, and the technique 
which aligns and displays two or more windows is indicated by JP,5-324244,A. 
[0004] 

[Problem(s) to be Solved by the Invention] However, in order to arrange a window in the case of the 
approach indicated by JP,5-324244,A, it is necessary to perform intentional actuation of choosing the 
command displayed on the display using the mouse. 

[0005] People perform actuation which usually brings a face close to an object, when checking an 
object, or when it is going to search. For example, when it is going to check the display in the window 
displayed on the display, or when it is going to look for the window hidden by other windows, actuation 
which brings a face close to a display is performed. Since it becomes unnecessary to carry out 
intentional actuation of choosing the command displayed on the display using the mouse when it 
becomes possible to change and arrange the display condition of a window, in order to know what kind 
of window is displayed on the display by detecting a natural motion of such a person, working 
efficiency improves. 

[0006] Then, this invention detects automatically a natural motion which the user of a display says is 
going to look for display elements, such as a window displayed on the display, and aims at offering the 
display control unit which changes the display condition of a display element so that it can know easily 
what kind of display element is displayed on the display. 
[0007] 

[Means for Solving the Problem] This invention is equipped with the following configurations as above- 
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mentioned The means for solving a technical problem. 

[0008] (1) Carry out having an image pick-up means picturize the location in which the user of a display 
is located, an extract means extract a specific image from the image picturized and obtained, a judgment 
means judge the migration condition of the extracted image, and a means change the display condition 
of two or more display elements displayed on the display based on the judgment result of a judgment 
means as the description. 

[0009] The migration condition of the image which extracted the specific image with the extract means 
and was extracted judges with a judgment means from the image which picturizes the location in which 
the user of a display is located with an image pick-up means, and is obtained in this configuration, and a 
display condition changes by means change the display condition of two or more display elements 
displayed on the display based on the judgment result of a judgment means. Therefore, when the user of 
a display is extracted as a specific image picturized with the image pick-up means, it becomes possible 
to change the display condition of display elements, such as a window displayed on the display by 
actuation of a display user. 

[0010] (2) Said extract means is characterized by extracting some bodies as a specific image. 
[0011] In this configuration, some bodies of the user of a display are extracted from the image picturized 
with the image pick-up means, and the display condition of display elements, such as a window which 
judged that migration condition and was displayed on the display, is changed. Therefore, the display 
condition of a display element can be changed by judging some migration conditions of the bodies, such 
as a head and the face. 

[0012] (3) Said judgment means makes it a criterion for the extracted specific image to have moved to 
the position of an image field. 

[0013] It judges with a judgment means that the specific image extracted from the image picturized and 
obtained with an image pick-up means with the extract means moved the location in which the user of a 
display is located to the position of an image field in this configuration, and a display condition changes 
by means change the display condition of two or more display elements displayed on the display based 
on the judgment result of a judgment means. Therefore, the display condition of a display element can 
be changed by actuation which is referred to as looking into the display of a display in order to look for 
an object. 

[0014] (4) Said judgment means makes it a criterion for the movement magnitude per fixed time amount 
of the extracted specific image to exceed the specified quantity. 

[0015] It judges with a judgment means that the movement magnitude per fixed time amount of the 
specific image extracted with the extract means exceeds the specified quantity from the image which 
picturizes the location in which the user of a display is located in this configuration with an image pick- 
up means, and is obtained, and a display condition changes by means change the display condition of 
two or more display elements displayed on the display based on the judgment result of a judgment 
means. Therefore, in order to check an object, actuation of looking into the display of a display is 
detected, and it becomes possible to change the display condition of a display element. 
[0016] (5) It is characterized by the ability to change the display position of a display element according 
to the migration direction of a specific image. 

[0017] The migration direction of the specific image extracted with the extract means detects from the 
image which picturizes the location in which the user of a display is located in this configuration with an 
image pick-up means, and is obtained, it judges with a judgment means, and a display condition changes 
by means change the display condition of two or more display elements displayed on the display based 
on the judgment result of a judgment means. Therefore, a display condition can be changed according to 
actuation of the user of a display, and it becomes possible to register two or more modification 
approaches of a display condition, and to use them properly. 

[0018] (6) Two or more display elements displayed on the display are characterized by changing a 
display condition so that it can carry out visible [ of the name display of a display element ]. 
[0019] The migration condition of the image which extracted a specific image with an extract means and 
extracted judges with a judgment means from the image which picturizes the location in which the user 
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of a display is located with an image pick-up means, and is obtained in this configuration, and a display 
condition is changed by means change the display condition of two or more display elements displayed 
on the display based on the judgment result of a judgment means so that the name display of a display 
element may carry out visible. Therefore, since a display condition is changed so that name displays, 
such as a title bar of display elements, such as an overlapping window, can be checked, the window 
currently displayed on the display can be grasped by glimpse. 

[0020] (7) In said extract means, if it becomes impossible to extract a specific image, it will be 
characterized by changing the display condition of a display. 

[0021] In this configuration, if it becomes impossible to extract a specific image from the image which 
picturizes the location in which the user of a display is located with an image pick-up means, and is 
obtained, the display condition of a display will be changed. Therefore, when a user stops being from 
before a display, it detects that a user is not, and the power source of a display can be turned off or a 
screen saver can be displayed on a display. 

[0022] (8) It is characterized by displaying the image picturized with the image pick-up means on a 
display. 

[0023] In this configuration, the image which picturizes the location in which the user of a display is 
located with an image pick-up means, and is obtained is displayed on a display. Therefore, since the 
image picturized with the image pick-up means is displayed directly on a display, the condition of the 
picturized image can be grasped easily. 

[0024] (9) It is characterized by making the image which picturized with the image pick-up means and 
was displayed on the display into the icon for starting of a means which changes the display condition of 
a display element. 

[0025] In this configuration, it considers as the icon for starting of a means which changes the display 
condition of the display element which displays on a display the image which picturizes the location in 
which the user of a display is located with an image pick-up means, and is obtained as an icon, and is 
displayed on a display in that icon. Therefore, since the image picturized with image pick-up means, 
such as a CCD camera, is an icon, it can check easily whether a display element is arranged by the 
migration situation of a specific image. 

[0026] (10) It judges that the image which extracted the specific image and was extracted moved from 
the picturized image data, and is characterized by changing the display condition of two or more display 
elements displayed on the display based on the judgment. 

[0027] In this configuration, the migration condition of the image which extracted the specific image 
and was extracted is judged from the image data which picturizes the location in which the user of a 
display is located and is obtained, and the display condition of two or more display elements displayed 
on : the display is changed based on the judgment result of a judgment means. Therefore, actuation of the 
user of a display is detected, since it becomes possible to change the display condition of display 
elements, such as a window displayed on the display, by actuation of a user, the display condition of a 
display element can be changed and arrangement etc. can be performed. 

[0028] (11) Carry out having recorded the display control program included in the processing which 
picturizes the location in which the user of a display is located, the processing which extract a specific 
image from the image data picturized and obtained, the processing which judge that the extracted image 
moved, and the processing which change the display condition of two or more display elements 
displayed on the display based on the judgment result as the description. 

[0029] In this configuration, the display control program carry out the processing which judges the 
migration condition of the image processed and extracted extract a specific image from the image data 
which picturizes the location in which the user of a display is located, and which is picturized [ is 
processed and ] and obtained, and the processing which change the display condition of two or more 
display elements displayed on the display based on the judgment result is recording to a record medium. 
Therefore, if the record medium with which this display control program was recorded is performed 
with a multi-window control device etc., the display condition of display elements, such as a window 
displayed on the display, can be changed by actuation of the user of a display. 
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[0030] 

[Embodiment of the Invention] Drawing 1 shows the example of a configuration of the device 
concerning the operation gestalt of this invention. It is a configuration containing the display 3 which 
displays the image outputted from the various results of an operation performed by the body 2 of a 
personal computer which is the multi-window control unit which performs various data processing in 
this drawing, and the body 2 of a personal computer, or the body 2 of a personal computer, and CCD 
camera 1 which is an image pick-up means. CCD camera 1 is installed in the center section on the top 
face of a case of a display 3. CCD camera 1 is used in order to input the image for recognizing actuation 
of the user who sits down in front of a display 3 in order to use the body 2 of a personal computer, and a 
display 3. Moreover, in the body 2 of a personal computer, it is possible to write a record medium. 
[0031] Drawing 2 is the block diagram showing the device shown in drawing 1 , and the image- 
processing section. CCD camera 1 which is an image pick-up means to picturize the user who sits down 
in front of a display 3 is connected to the image memory section 11 of the image-processing section 10 
within the body 2 of a personal computer. Moreover, the display and control section 17 which controls 
the image data displayed on a display 3 is connected to the movement magnitude judging section 16 of 
the image-processing section 10. Furthermore, the display 3 is connected to the display and control 
section 17. It connects in order of the image memory section 11, the image feature-extraction section 12, 
the image description memory section 18, the center-of -gravity count section 14, the time series memory 
section 15 of a center of gravity, and the movement magnitude judging section 16, and the image- 
processing section 10 is constituted. Moreover, the criteria data division 19 are connected to the 
movement magnitude judging section 16. Here, each part is controlled by CPU20. Each part is 
connected to CPU20 although not illustrated in drawin g 2 . 

[0032] The image-processing section 10 of this invention explains by mentioning as an example the case 
where the image of the face of the user of a display 3 is extracted as a specific image. If a user sits down 
in front of a display 3 in order to use the body 2 of a personal computer, and a display 3, a user's upper 
half of the body and background will be picturized by CCD camera 1. The image picturized by CCD 
camera 1 is temporarily recorded on the image memory section 11. Next, in the image feature-extraction 
section 12, the image of a user's face is extracted from the image data recorded on the image memory 
section 11. Under the present circumstances, paying attention to a beige field with changing [ little ] 
sharply in an image, that field judges with it being the image of the face, and is extracted as an easy 
graphic form. Here, it is extracted by the ellipse form as an example of an easy graphic form. 
[0033] The extracted image is recorded on the image description memory section 18. Next, in the center- 
of-gravity count section 14, count of the center of gravity of the extracted face image is performed. 
Since what is necessary is just to calculate the movement magnitude of a center of gravity if the center 
of gravity of a face image is calculated, count of the movement magnitude of a face image can be 
simplified. The center of gravity at the time of moving is recorded on the time series memory section 15 
of a center of gravity for every predetermined time amount. The contents of the time series memory 
section 15 of a center of gravity and the contents of the criteria data division 19 are compared by the 
movement magnitude judging section 16, and it is judged whether display conditions which are the 
display elements displayed on the display 3, such as a window and an icon, are changed. Drawin g 5 and 
explanation of 6 describe the decision criterion of modification here in detail. If the judgment which 
changes display conditions, such as a window, is made, a change of a display condition will be made by 
the display and control section 17, and the screen where display conditions, such as a window, were 
changed into the display 3 will be displayed. 

[0034] Drawin g 3 is the image picturized by CCD camera 1. the user 31 photoed by CCD camera 1 in 
this drawing - an image - it has projected in the center mostly. When working using the body 2 of a 
computer, looking at a display 3, since a user 31 usually sits down in front of a display 3, a background 
32'is picturized with a user 31 as an image as shown in this drawing. 

[0035] Drawin g 4 is drawing showing the center of gravity of the image of the user of the display 3 
picturized by CCD camera 1, and the image of the extracted face. As shown in drawin g 4 , when it 
moves to right-hand side to CCD camera 1, the center of gravity of the image of the face is moved to a 
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center of gravity 42 from a center of gravity 41. Moreover, when it moves to left-hand side to CCD 
camera 1, the center of gravity of the image of the face is moved to a center of gravity 43 from a center 
of gravity 41. As mentioned above, since it is necessary to perform huge count if the movement 
magnitude of the whole **** is calculated when a user moves, it calculates noting that the center of 
gravity is moving in simple. 

[0036] Drawin g 5 is the example which changed the display condition of two or more windows 
displayed on the display. Drawing 5 shifts each windows 71-74 little by little, and is displayed in piles 
so that the display condition shown in drawing 1515 may be changed and the title bars 71b-74b which 
are name displays so that the name of the window displayed on the display 3 can be looked through can 
be seen. 

[0037] At this time, the sequence which displays in piles the position and window in the screen which 
puts a window in order enables a setup to arbitration by the user. And the sequence which displays a 
window in piles has sequence, such as started time amount as which the overlapping order and a 
window were displayed, and time amount operated at the last. Moreover, when displaying a window in 
piles in such sequence, there are an approach of arranging in ascending order and the approach of 
arranging in descending order about each sequence. 

[0038] Drawin g 6 is another example which changed the display condition of two or more windows 
displayed on the display 3. In drawin g 6 , the display condition shown in drawing 15 is changed, and 
each windows 71-74 are displayed on the same magnitude, and are put in order and displayed on the 
screen of a display 3. 

[0039] Drawin g 7 is drawing showing how to make it into the criterion which changes the display 
element displayed [ that the image of the face extracted as a specific image moved to the position, and ] 
on the display. This approach is an approach of changing display conditions, such as a window, when 
the image of the face moves into the predetermined image field appointed beforehand. In this drawing, 
when the center of gravity 41 of the image of a face moves to the field 54 on the right of the right-hand 
side boundary line 51, or the field 55 on the left of the left-hand side boundary line 52, display 
conditions, such as a window, are changed. It is possible to also carry out a setup of the modification 
approach of display conditions, such as a window, to be alike identically and to also make [ the case 
where a center of gravity 41 moves to the location of the center of gravity 44 of the field 54 on the right 
of the right-hand side boundary line 51 by this approach, the case where a center of gravity 41 moves to 
the location of the center of gravity 45 of the field 55 on the left of the left-hand side boundary line 52, 
and ] it an option. For example, it can change into the display shown in drawing 5 from the display 
shown in drawing 15 in the case where a center of gravity 41 moves to the location of the center of 
gravity 44 of the field 54 on the right of the right-hand side boundary line 51, the case where a center of 
gravity 41 moves to the location of the center of gravity 45 of the field 55 on the left of the left-hand 
side boundary line 52, and both cases. Moreover, when it changes into the display shown in drawin g 5 
R> 5 from the display shown in drawing 15 when a center of gravity 41 moved to the location of the 
center of gravity 44 of the field 54 on the right of the right-hand side boundary line 51 and a center of 
gravity 41 moves to the location of the center of gravity 45 of the field 55 on the left of the left-hand 
side boundary line 52, it can also change into the display shown in drawin g 6 from the display shown in 
drawing 15 . 

[0040] Drawin g 8 is drawing showing how to make it a criterion for the movement magnitude by which 
the image of the face extracted as a specific image ran in fixed time amount to exceed the specified 
quantity. By this approach, the migration length 53 between the center of gravity 41 of the image of the 
first face and the center of gravity 46 of the image of the face after predetermined time amount judges 
with having moved, when the specified quantity was exceeded in predetermined time amount. It is 
possible to also carry out a setup of the modification approach of display conditions, such as a window, 
to be alike identically and to also make [ the case where the direction which moved by this approach is 
right-hand side, the case where the direction which moved is left-hand side, and ] it an option. 
[0041] The user of a display may remove a seat on a certain business. It becomes impossible in that 
case, to picturize the image of the face with which the user was extracted by the display with a CCD 
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camera. Since it is considered that the user stopped using a display temporarily when it becomes 
impossible for the image of the face of the user of a display to picturize with a CCD camera, it becomes 
unnecessary to display on a display. Therefore, the power source of a display is turned off or a screen 
saver is displayed on a display. CCD camera 1 is operated at this time, and a user is returned to the 
condition if the image of return and the face is picturized by CCD camera 1, before, as for the display of 
a display 3, it will remove a seat automatically before a display 3 by finishing business. Moreover, it can 
also set up so that display conditions, such as a window displayed on the screen, may be changed. 
[0042] Drawing 9 is the example which set to icon 81a the image picturized with CCD camera 1, and 
was displayed on the upper right portion of a display 3. As shown in drawing 9 , when the image 
picturized with CCD camera 1 is displayed on a display 3, it is turned out how the user 31 is picturized 
at a glance. Moreover, it does not become obstructive in case other windows will be operated, if it 
displays on the edge of Screen 82 of a display 3. Furthermore, if this icon 81a is clicked, effective and 
the invalid of processing which changes the display condition of a display element can be changed. 
When effective, icon 81a is trimmed. In the case of an invalid, icon 81a is not trimmed, but drops the 
brightness of the image which CCD camera 1 photoed, and is displayed. Thus, when it displays, 
effective and the invalid of processing which changes a display condition are known at a glance. In 
addition, effective and an invalid may be displayed not only by the above-mentioned approach but by 
the option. 

[0043] Drawing 10 is the example of the icon displayed on a display 3 as a start switch of a system 
which changes the display condition of a window, when the image of the face explained using drawin g 7 
moves into a predetermined field. The boundary line 83 of the field where icon 81b shown in drawing 

10 becomes the conditions which make a display change of a window is displayed. If icon 81b is 
checked, and it moves how far, it can grasp easily whether a change of the display condition of a 
window is made. 

[0044] Drawing 11 is the example of the icon displayed on a display as a start switch of a system which 
changes the display condition of a window for the image of the face explained using drawin g 8 having 
moved exceeding the specified quantity into predetermined time amount. Two or more indication of the 
line 84 which shows the predetermined movement magnitude from which icon 81c shown in drawing 1 1 
becomes the conditions which make a display change of a window is given at equal intervals. Moreover, 
predetermined time amount is displayed on the upper left section of icon 81c. It is displayed on drawin g 

11 as 0.5 seconds as an example of predetermined time amount. If icon 81c is checked, it can grasp 
easily with the movement magnitude like which a change of the display condition of a window is made. 
[0045] Drawing 12 is a flow chart which shows the procedure which changes the display condition of a 
window. Drawing 13 is a flow chart which shows the procedure which changes the display condition of 
a window, when the image of the face explained using drawin g 7 moves into a predetermined field. 
Drawing 14 is a flow chart which shows the procedure which changes the display condition of a 
window, when the image of the face explained using drawin g 8 moves exceeding the specified quantity 
into predetermined time amount. Processing of drawing 13 or drawing 14 is performed at the step (s4) 
which processes the flow chart shown in drawing 12 . 

[0046] In drawing 12 , starting of the operation system of a personal computer displays (si) and the 
image picturized with CCD camera 1 as an icon 81 on a display 3 (s2). A click of the icon 81 starts the 
system which changes the display condition of a window (s3). 

[0047] If the image of the face moves into a predetermined field, when performing processing which 
changes the display condition of a window (s4), if processing is started (sll), in drawing 13 , the image 
of the face which is a specific image will be extracted first (si 2). The center of gravity is calculated from 
the image of the extracted face (si 3), and it is confirmed whether it moved to (s20) and a predetermined 
field by comparing with a reference value (si 4). Processing is ended if it is not moving into a here 
predetermined field (sl7). If the center of gravity of a face image is moving into a predetermined field, 
modification processing of the display condition of a window will be performed (si 5). The result is 
outputted to a display 3 (sl6), and processing is ended (sl7). in addition, the time of performing the 
check of how, although it moved to the predetermined field - (si 4) and right and left -- the arrangement 
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approach of the window performed in the part of modification processing (si 5) of the display condition 
of a window can be changed by to which field it moved. 

[0048] Moreover, in drawin g 14 , if the center of gravity of the image of the face moves exceeding the 
specified quantity into predetermined time amount, when changing the display condition of a window, if 
processing is started (sll), the image of the face which is a specific image will be extracted first (si 2). 
The center of gravity is calculated from the image of the extracted face, and the movement magnitude is 
calculated as compared with the coordinate of the center of gravity in front of fixed time amount (sl8). 
Processing is ended if it is not this movement magnitude over the reference value as compared with the 
reference value (s20) (si 9) (si 7). If it is over the reference value (si 9), modification processing of the 
display condition of a window file will be performed (si 5). in addition, the direction which moved when 
checking, whether movement magnitude is over the reference value, and — right and left — either is 
judged and the file arranging method is changed. 

[0049] In drawing 12 , if a user clicks on an icon 81 and chooses termination (s5), a post process will be 
performed (s8) and a system will be ended. When there is no image of the face which is a specific image 
into the image picturized by CCD camera 1, without a user choosing termination (s5) (s6), the power 
source of a display 3 is turned off or a screen saver is displayed on a display 3 (s7). If the image pick-up 
of an image is continued (s6) and the image of the face is again picturized by CCD camera 1 even if the 
display condition of a display 3 is changed, the display of a display 3 will be returned. When the image 
of the face is in the image picturized by CCD camera 1 (s6) and the display condition of a display 3 is 
changed (i.e., when the power source of a display 3 turns off or the screen saver is displayed on the 
display 3) (s9), the display of a display 3 is returned (slO) and processing explained in drawing 13 or 
drawing 1414 is performed (s4). The image of the face is in the image picturized by CCD camera 1 (s6), 
and when the display condition of a display 3 is not changed (s9), processing explained by drawin g 13 
or drawing 14 is performed (s4). 

[0050] Here, if the display control program which performs the above-mentioned processing is recorded 
on a record medium, it will become possible to perform the program recorded on the record medium 
with other personal computers. 

[0051] As mentioned above, although the personal computer which is a multi- window control device 
was mentioned as the example and this invention was explained, there is a television set as other 
examples. 

[0052] In drawin g 1 , when the receiving set of television is contained by the body 2 of a personal 
computer, the image of the channel which received a picture can be displayed on a display 3. In 
addition, the receiver 2 of television may be built in the display 3. 

[0053] In the latest television set, there is multichannel mode which displays the image of two or more 
received channels on a display at once. That is, when displaying the image of the channel in which 
current reception is possible on a display, choosing a favorite channel and displaying only the image of 
that channel, this mode display is performed. In case a favorite channel is chosen, the carbon button of 
the remote controller outside drawing and the control unit of a display are operated, and a channel is 
chosen. 

[0054] In this invention, in case the image of a favorite channel is chosen from the image of two or more 
channels displayed on the display 3, CCD camera 1 detects bringing a face close in the image of the 
favorite channel displayed on the display 3, and a channel is chosen as it. Thus, the user of a display 3 
can detect automatically the actuation which is going to choose display elements, such as a channel 
displayed on the display 3, can change the display condition of the display element of a display 3, and 
can display only the image of a favorite channel. 

[0055] In addition, as an example of this invention, although the multi-window control device and the 
television set were mentioned, this invention is not restricted to this example. Moreover, although the 
face image of the user of a display was explained as a specific image when the personal computer which 
is a multi-window control device was mentioned as an example and explained, it does not restrict to this. 

[0056] 
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[Effect of the Invention] According to this invention, the following effectiveness is acquired. 
(1) An extractmeans extracts a specific image from the image which picturizes the location in which the 
user of a display is located with an image pick-up means, and is obtained. With a means to change the 
display condition of two or more display elements which judged the migration condition of the extracted 
image with the judgment means, and were displayed on the display based on the judgment result of a 
judgment means When the user of a display is extracted as a specific image picturized with the image 
pick-up means by changing a display condition, It becomes possible to change the display condition of 
display elements, such as a window displayed on the display by actuation of a display user, and by 
natural actuation of bringing a face close to an object, since display conditions, such as a window, can 
be changed, working efficiency improves. 

[0057] (2) The display condition of a display element can be changed by judging some migration 
conditions of the bodies, such as a head and the face, by changing the display condition of display 
elements, such as a window which extracted some bodies of the user of a display from the image 
picturized with the image pick-up means, judged the migration condition, and was displayed on the 
display. 

[0058] (3) The specific image which extracted the location in which the user of a display is located with 
the extract means from the image picturized and obtained with an image pick-up means With a means to 
change the display condition of two or more display elements which judged having moved to the 
position of an image field with the judgment means, and were displayed on the display based on the 
judgment result of a judgment means in it By changing a display condition, the display condition of a 
display element can be changed by actuation which is referred to as looking into the display of a display 
in order to look for an object. 

[0059] (4) From the image picturized and obtained with an image pick-up means, the location in which 
the user of a display is located With a means to change the display condition of two or more display 
elements which judged with the judgment means that the movement magnitude per fixed time amount of 
the specific image extracted with the extract means exceeded the specified quantity, and were displayed 
on the display based on the judgment result of a judgment means in it By changing a display condition, 
in order to check an object, actuation of looking into the display of a display is detected, and it becomes 
possible to change the display condition of a display element. 

[0060] (5) From the image picturized and obtained with an image pick-up means, the location in which 
the user of a display is located With a means to change the display condition of two or more display 
elements which detected the migration direction of the specific image extracted with the extract means, 
judged with the judgment means, and were displayed on the display based on the judgment result of a 
judgment means By changing a display condition, a display condition can be changed according to 
actuation of the user of a display, and it becomes possible to register two or more modification 
approaches of a display condition, and to use them properly. 

[0061] (6) An extract means extracts a specific image from the image which picturizes the location in 
which the user of a display is located with an image pick-up means, and is obtained. With a means to 
change the display condition of two or more display elements which judged the migration condition of 
the extracted image with the judgment means, and were displayed on the display based on the judgment 
result of a judgment means Since a display condition is changed so that it can carry out visible [ of the 
name display of a display element ], and name displays, such as a title bar of display elements, such as a 
window which lapped by changing a display condition, can be checked, the window currently displayed 
on the display can be grasped by glimpse. 

[0062] (7) If it becomes impossible to extract a specific image from the image which picturizes the 
location in which the user of a display is located with an image pick-up means, and is obtained, the 
display condition of a display will be changed. Therefore, since it detects that a user is not, and the 
power source of a display can be turned off or a screen saver can be displayed on a display when a user 
stops being from before a display, there is the energy-saving effectiveness. 

[0063] (8) Since the image picturized with the image pick-up means by displaying on a display the 
image which picturizes the location in which the user of a display is located with an image pick-up 
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means, and is obtained is displayed directly on a display, the condition of the picturized image can be 
grasped easily. 

[0064] (9) Since the image which picturized with image pick-up means, such as a CCD camera, by 
considering as the icon for starting of a means which changes the display condition of the display 
element which displays on a display the image which picturizes the location in which the user of a 
display is located with an image pick-up means, and is obtained as an icon, and is displayed on a display 
in the icon is an icon, it can check [ whether a display element is arranged by the migration situation of a 
specific image, and ] easily. 

[0065] (10) Extract a specific image from the image data which picturizes the location in which the user 
of a display is located and is obtained. By judging the migration condition of the extracted image and 
changing the display condition of two or more display elements displayed on the display based on the 
judgment result of a judgment means Actuation of the user of a display is detected, since it becomes 
possible to change the display condition of display elements, such as a window displayed on the display, 
by actuation of a user, the display condition of a display element can be changed and arrangement etc. 
can be performed. 

[0066] (11) The processing which extracts a specific image from the image data which picturizes the 
location in which the user of a display is located, and which is picturized [ is processed and ] and 
obtained, By recording the display control program which performs processing which judges the 
migration condition of the extracted image, and processing which changes the display condition of two 
or more display elements displayed on the display based on the judgment result on a record medium If 
the record medium with which this display control program was recorded is performed with a multi- 
window control device etc., the display condition of display elements, such as a window displayed on 
the display, can be changed by actuation of the user of a display. 



[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

l.This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **** shows the word which can not be translated. 
3 .In the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 11 The example of a configuration of the device concerning the operation gestalt of this 
invention is shown. 

[Drawing 21 It is the block diagram of the image-processing section concerning the operation gestalt of 
this invention. 

[Drawin g 31 It is drawing having shown the image picturized by the CCD camera. 

[Drawing 41 It is drawing showing the center of gravity of the image picturized by the CCD camera, and 

the image of the extracted face. 

[Drawing 51 It is the example which changed the display condition of two or more windows displayed 
on the display. 

[Drawing 61 It is another example which changed the display condition of two or more windows 
displayed on the display. 

[Drawing 71 The image of the face extracted as a specific image is drawing showing how to make it a 
criterion whether to move to a position. 

[Drawing 81 It is drawing showing how to make it a criterion whether for the movement magnitude by 
which the image of the face extracted as a specific image ran in fixed time amount to have exceeded the 
specified quantity. 

[Drawing 91 It is the example displayed on the upper right portion of a display by making into an icon 
the image picturized with the CCD camera. 

[Drawing 101 When the image of the face moves into a predetermined field, it is the example of the icon 
displayed on a display as a start switch of a system which changes the display condition of a window. 
[Drawing 111 It is the example of the icon displayed on the display at the time of making for the image 
of the face to have moved exceeding the specified quantity into predetermined time amount into the start 
switch of a system which changes the display condition of a window. 

[Drawing 121 It is the flow chart which shows the procedure in the case of changing the display 
condition of a window. 

[Drawing 131 When the image of the face moves into a predetermined field, it is the flow chart which 
shows the procedure in the case of changing the display condition of a window. 
[Drawing 141 When the image of the face moves exceeding the specified quantity into predetermined 
time amount, it is the flow chart which shows the procedure in the case of changing the display 
condition of a window. 

[Drawin g 151 Usually, it is the example which shows the condition of the window displayed on the 
display currently used. 
[Description of Notations] 

I- CCD camera 
3-display 

10-image-processing section 

II - image memory section 
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# 



12-images feature-extraction section 

14-center-of -gravity count section 

The time series memory section of 15-center of gravity 

16- movement magnitude judging section 

17- display and control section 
18 : images description memory section 
19-criteria data division 



[Translation done.] 
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